Writing Stories with Help from Recurrent Neural Networks

Automated story generation has a long history of pursuit in artificial intelligence. Early approaches used hand-authored formal models of a particular story-world domain to generate narratives pertaining to that domain (Klein, Aeschlimann, and Balsiger 1973; Lebowitz 1985; Meehan 1977). With the advent of machine learning, more recent work has explored how to construct narrative models automatically from story corpora (Li et al. 2013; McIntyre and Lapata 2009; Swanson and Gordon 2012). This research has created a new potential for interactivity in narrative generation. Unlike previous approaches which lacked the breadth of knowledge required for open-domain storytelling, these systems leverage story data to interface with authors pursuing diverse narrative content. For example, Swanson and Gordon (Swanson and Gordon 2012) demonstrated an application where a user and automated agent took turns contributing sentences to a story. Their system used a case-based reasoning approach to retrieve a relevant continuation of the user’s sentence from a large database of stories. This research has given rise to a new type of story generation task, one of “narrative auto-completion”, where a system analyzes an ongoing narrative and generates a new contribution to the story. Analogous to existing automated writing aids like spelling and grammar correction, narrative auto-completion is applicable as a writing tool that suggests new ideas to authors. Recurrent Neural Networks (RNN) are a promising machine learning framework for language generation tasks. In natural language processing (NLP) tasks, RNNs are trained on sequences of text to model the conditional probability distribution of predicting a sequence unit (often a character or word) given the sequence up to that point. After training it is straightforward to generate new text by iteratively predicting the next unit based on the text generated so far. In this same way, a given text can be extended by predicting additional text in the sequence. For this reason an RNN is a suitable engine for an automated story writing assistant that takes an ongoing story as input for predicting a continuation of the story. In this thesis I explore the use of RNNs for this novel generation task, and show how this task affords a unique opportunity for the evaluation of generation systems.